Composite image generation apparatus and composite image generation program

ABSTRACT

In a composite image generation apparatus that is mounted in an own vehicle, each of captured images, which have been captured by a plurality of imaging units, are acquired. A disturbance level for each of the plurality of captured images that have been acquired is determined. The disturbance level indicates: whether or not a disturbance is present in the captured image; or an extent of the disturbance. In an overlapping area in which imaging areas of the plurality of captured images overlap, one or more captured images is selected from the plurality of captured images so that the area occupied by a captured image having a higher disturbance level among the plurality captured images is smaller. A composite image is generated based on the plurality of captured images. In the overlapping area, the composite image is generated by using the captured image selected based on the disturbance level.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims the benefit of priority fromJapanese Patent Application No. 2014-048892, filed Mar. 12, 2014, thedisclosure of which is incorporated herein in its entirety by reference.

BACKGROUND

Technical Field

The present disclosure relates to a composite image generation apparatusand a composite image generation program for generating a compositeimage that is a combination of a plurality of images.

Related Art

As the above-described composite image generation apparatus, thefollowing known (for example, refer to JP-A-2012-134586). The compositeimage generation apparatus acquires captured images of a vehicleperiphery. These images have been captured by a plurality of cameras.Based on the captured images, the composite image generation apparatusthen generates a bird's-eye view image that is supposed to show a viewfrom directly above the vehicle (own vehicle). In this composite imagegeneration apparatus, when imaging areas of the plurality of capturedimages overlap, the captured images are cut at sections that have beenset in advance. Portions of the overlapping areas are deleted. Theimages are then joined together, thereby forming a natural-lookingbird's-eye view image.

The above-described birds-eye view image (composite image) is used tocheck the periphery of the own vehicle. However, for example, a stronglight may be captured in any of the captured images. In such instances,blocked up shadows (a phenomenon in which an area other than a lightsource in a captured image becomes black) may be formed in the capturedimage. The area over which the periphery can be checked using thecomposite image becomes smaller.

In addition, similar situations occur when dirt, water drops, and thelike attach to the camera lens. In such instances, a problem occurs inthat it becomes difficult to check the periphery using the compositeimage.

SUMMARY

It is thus desired to provide a composite image generation apparatusthat is mounted in an own vehicle and generates a composite image thatis a combination of captured images acquired by a plurality of imagingunits, in which a composite image can be generated that facilitateschecking of the periphery of the own vehicle.

An exemplary embodiment of the present disclosure provides a compositeimage generation apparatus that includes captured image acquiring means,disturbance level determining means, captured image selecting means, andimage generating means. The captured image acquiring means acquires eachof captured images that have been captured by a plurality of imagingunits. The disturbance level determining means determines a disturbancelevel for each of the plurality of captured images that have beenacquired. The disturbance level indicates: whether or not a disturbanceis present in the captured image; or an extent of the disturbance. Thecaptured image selecting means selects, in an overlapping area in whichimaging areas of a plurality of captured images overlap, one or morecaptured images from the plurality of captured images so that the areaoccupied by a captured image having a higher disturbance level among theplurality captured images is smaller. The image generating meansgenerates a composite image based on the plurality of captured images,and generates the composite image by using the captured image selectedbased on the disturbance level in the overlapping area.

In a composite image generation apparatus such as this, the proportionin a composite image occupied by a captured image in which a disturbancehas occurred can be reduced based on the presence of a disturbance andthe extent of the disturbance. Therefore, a composite image can begenerated that facilitates checking of the periphery of an own vehicle.

In the present disclosure, a composite image generation program may beprovided that enables a computer to actualize each means configuring thecomposite image generation apparatus. In addition, the expressions inthe claims can be arbitrarily combined to the extent possible. In thiscase, some configurations may be eliminated to an extent allowing theobject of the present disclosure to be achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a block diagram of an overall configuration of an imagedisplay system to which the present disclosure is applied;

FIG. 2 is a birds-eye view of camera placement positions;

FIG. 3 is a flowchart of a white-line recognition process performed byan image processing unit (CPU);

FIG. 4 is a flowchart of a disturbance determination process in thewhite line recognition process;

FIG. 5A to FIG. 5C are bird's-eye views of an example of captured imagesand overlapping visual field areas;

FIG. 6 is a flowchart of a composite birds-eye view generation processin the white-line recognition process;

FIG. 7 is a birds-eye view showing an example of a composite birds-eyeview image (1);

FIG. 8A and FIG. 8B are birds-eye views showing an example of acomposite birds-eye view image (2); and

FIG. 9A and FIG. 9B are birds-eye views showing an example of acomposite birds-eye view image (3).

DESCRIPTION OF EMBODIMENTS

An embodiment of the present disclosure will hereinafter be describedwith reference to the drawings.

[Configuration]

The present disclosure is applied to an image display system 1, as shownin FIG. 1. The image display system 1 is mounted in a vehicle, such as apassenger car (also referred to, hereafter, as an own vehicle). Theimage display system 1 generates a composite image that is a combinationof captured images acquired by a plurality of cameras 21 to 24. Theimage display system 1 recognizes road surface paint, such as whitelines, from the generated image.

Specifically, as shown in FIG. 1, the image display system 1 includes animage processing unit 10, various cameras 21 to 24, a display 26, and anindicator 27. The various cameras 21 to 24 include a front camera 21, arear camera 22, a right camera 23, and a left camera 24. The cameras 21to 24 each have an imaging area, as shown by the shaded portions in FIG.2 and the like. In FIG. 2 and the like, each imaging area isschematically shown using a fan shape (half circle). However, the actualimaging area does not necessarily take this shape. The imaging area maybe an arbitrary shape and size.

More specifically, the front camera 21 is disposed, for example, in thefront bumper. The imaging area of the front camera 21 is the area aheadof the own vehicle VE. The rear camera 22 is disposed, for example, inthe rear bumper. The imaging area of the rear camera 22 is the areabehind the own vehicle VE.

The right camera 23 is disposed, for example, in the right side mirrorof the own vehicle VE. The imaging area of the right camera 23 is thearea to the right of the own vehicle VE. The left camera 24 is disposed,for example, in the left side mirror of the own vehicle VE. The imagingarea of the left camera 24 is the area to the left of the own vehicleVE. For example, each camera 21 to 24 captures an image every 33milliseconds (ms). The cameras 21 to 24 then send the captured images tothe image processing unit 10.

The display 26 is configured as a known display that displays imagesbased on image signals sent from the image processing unit 10.

The indicator 27 gives visual notification based on a display commandfrom the image processing unit 10. For example, the indicator 27 givesnotification regarding the degree of recognition accuracy of roadsurface paint, such as white lines. The indicator 27 includes, forexample, a plurality of light-emitting units. The number oflight-emitting units that are lit changes depending on the recognitionaccuracy. The recognition accuracy of the road surface paint indicatesthe accuracy of a white-line extraction process (step S135), describedhereafter. The image processing unit 10 performs output based on thisaccuracy.

The image processing unit 10 is configured by a known computer thatincludes a central processing unit (CPU) 11 and a memory 12, such as aread-only memory (ROM) or a random access memory (RAM). The CPU 11performs various processes, such as a white-line recognition process,described hereafter, based on programs (including a composite imagegeneration program) stored in the memory 12.

[Processes]

In the image display system 1 configured as described above, the imageprocessing unit 10 performs the white-line recognition process shown inFIG. 3. The image processing unit 10 starts the white-line recognitionprocess when, for example, the power of the image processing unit 10 isturned ON. The image processing unit 10 then repeatedly performs thewhite-line recognition process at a fixed interval (such as every 33milliseconds).

In the white-line recognition process, first, the image processing unit10 acquires the captured images that have been captured by the pluralityof cameras 21 to 24 (step S110). The image processing unit 10 thenperforms a disturbance determination process (step S120). Thedisturbance determination process is performed to determine adisturbance level. The disturbance level indicates whether or not adisturbance is present in each captured image, or the extent of thedisturbance. Here, the type of disturbance is also determined as a partof the disturbance level.

In the disturbance determination process, as shown in FIG. 4, first, theimage processing unit 10 sets a variable n to 1 (step S210). In otherwords, the image processing unit 10 selects an n-th captured image.Here, the variable n designates any of the captured images. For example,the variable n is assigned as follows: the captured image by the frontcamera 21 is n=1; the captured image by the rear camera 22 is n=2; thecaptured image by the right camera 23 is n=3; and the captured image bythe left camera 24 is n=4.

Next, the image processing unit 10 determines whether or not a saturatedportion in which a brightness value is saturated is present in theselected n-th image (step S215). In this process, the image processingunit 10 determines the brightness value of each pixel composing thecaptured image. The image processing unit 10 makes an affirmativedetermination when the brightness value is saturated (a brightness levelof 100% is present).

When determined that a saturated portion in which a brightness value issaturated is not present (NO at step S215), the image processing unit 10proceeds to the process at step S225, described hereafter. Whendetermined that a saturated portion in which a brightness value issaturated is present (YES at step S215), the image processing unit 10records, in the memory 12, that a light source is present in thiscaptured image (step S220). The light source, such as the sun orheadlights, may adversely affect white-line recognition.

When a light source is present in the captured image in this way, forexample, the image processing unit 10 deducts 30 points from an allottedscore (such as an initial score of 100 points) of the captured image.The image processing unit 10 then records the score in the memory 12.The number of points deducted is an example. However, the amount ofdeduction increases for light sources (types of disturbances) presumedto have a greater adverse effect on the recognition of white lines fromthe captured image.

Next, the image processing unit 10 determines whether or not a blurredportion is present in the captured image (step S225). The blurredportion refers to an area that has a brightness value that is greaterthan that of “dirt (stains, smudges)”, described hereafter. In theblurred portion, the difference in brightness value between adjacentpixels is small. Also, the blurred portion is out of focus.

When determined that a blurred portion is not present in the capturedimage (NO at step S225), the image processing unit 10 proceeds to theprocess at step S250, described hereafter. When determined that ablurred portion is present in the captured image (YES at step S225), theimage processing unit 10 determines the size of the blurred portion(step S230).

When determined that the size of the blurred portion is a size thresholdor greater (YES at step S230), the image processing unit 10 determinesthat the size of the blurred portion is large. The size threshold is setin advance. The image processing unit 10 then records, in the memory 12,that a large rain drop (water drop) is present in the captured image(step S235). When a large rain drop is present in the captured image inthis way, for example, the image processing unit 10 deducts 30 pointsfrom the allotted score of the captured image. The image processing unit10 then records the score in the memory 12.

When determined that the size of the blurred portion is less than thesize threshold (NO at step S230), the image processing unit 10determines that the size of the blurred portion is small. The imageprocessing unit 10 then records, in the memory 12, that a small raindrop is present in the captured image (step S240). When a small raindrop is present in the captured image in this way, for example, theimage processing unit 10 deducts only 10 points from the allotted scoreof the captured image. The amount of deduction in this case is less thanthat when a light source or a large rain drop is present in the capturedimage. The image processing unit 10 then records the score in the memory12.

Next, the image processing unit 10 determines whether or not an areathat has a significantly low brightness value in relation to thebrightness values of the surrounding pixels is present (step S250). Whendetermined that an area having a significantly low brightness value isnot present (NO at step S250), the image processing unit 10 proceeds tothe process at S260, described hereafter.

When determined that an area having a significantly low brightness valueis present (YES at step S250), the image processing unit 10 records, inthe memory 12, that dirt is present in the image. When dirt is presentin the captured image in this way, for example, the image processingunit 10 deducts 20 points from the allotted score of the captured image.The image processing unit 10 then records the score in the memory 12(step S255). The number of points deducted in this case is set to begreater than that when a small rain drop is present in the capturedimage and less than that when a light source or a large rain drop ispresent in the captured image.

Next, the image processing unit 10 compares the variable n with thenumber N of the captured images (step S260). The number N of thecaptured images is four because four cameras 21 to 24 are presentaccording to the present embodiment. When determined that the variable nis less than the number N of the captured images (NO at step S260), theimage processing unit 10 increments the variable n. The image processingunit 10 then returns to the process at step S215. When determined thatthe variable n is the number N of the captured images or more, the imageprocessing unit 10 ends the disturbance determination process.

Next, the image processing unit 10 returns to the process in FIG. 3. Theimage processing unit 10 performs a composite birds-eye view conversionprocess (step S125). A geometric transformation table is provided forconverting each of the captured images to a birds-eye view (bird's-eyeview image) that is supposed to be viewed from directly above the ownvehicle VE. In the composite birds-eye view conversion process, theimage processing unit 10 uses the geometric transformation table toperform coordinate conversion on the pixels in the captured images. Theimage processing unit 10 thereby obtains captured images that have beenconverted to a bird's-eye view image.

For example, the own vehicle VE is traveling on a road, as shown in FIG.5A. The images obtained from the cameras 21 to 24 are converted. As aresult, the birds-eye view images shown in FIG. 5B and FIG. 5C areobtained. In other words, as shown in FIG. 5B, a front image 40 capturedby the front camera 21 is obtained. In addition, a rear image 45captured by the rear camera 22 is obtained. As shown in FIG. 5C, a rightimage 50 captured by the right camera 23 is obtained. In addition, aleft image 55 captured by the left camera 24 is obtained.

Next, the image processing unit 10 performs a composite bird's-eye viewimage generation process (step S130). In the composite birds-eye viewimage generation process, the image processing unit 10 obtains acomposite birds-eye view image (composite image) by combining thebird's-eye view images corresponding to the captured images.

Here, it is noted that overlapping visual field areas (overlappingareas) are present in the bird's-eye view images obtained by convertingthe captured images to bird's-eye view. The overlapping visual fieldarea refers to an area in which the imaging areas of a plurality ofcaptured images overlap. For example, an area 41 of the front image 40shown in FIG. 5B overlaps with an area 56 of the left image 55 shown inFIG. 5C. The area 41 is further to the left of the own vehicle VE. Thearea 56 is further ahead of the own vehicle VE.

In addition, in a similar manner, an area 42 of the front image 40 andan area 51 of the right image 50 form an overlapping visual field area.The area 42 is to the right of the own vehicle VE. The area 51 is aheadof the own vehicle VE. An area 46 of the rear image 45 and an area 57 ofthe left image 55 also form an overlapping visual field area. The area46 is to the left of the own vehicle VE. The area 57 is behind the ownvehicle VE. Furthermore, an area 47 of the rear image 45 and an area 52of the right image 50 also form an overlapping visual field area. Thearea 47 is to the right of the own vehicle VE. The areas 52 is behindthe own vehicle VE.

Therefore, in the composite birds-eye view image generation process, theimage processing unit 10 appropriately selects the image to be used inthe overlapping visual field area, taking disturbance intoconsideration, when generating the composite birds-eye view image.

The details of the composite birds-eye view image generation process areas follows. As shown in FIG. 6, first, the image processing unit 10acquires disturbance information of the captured images by the cameras21 to 24 (step S310). Here, the disturbance information refer to theinformation on the type of disturbance and the score that have beenrecorded in the memory 12 in the above-described disturbancedetermination process. In addition, in the composite bird's-eye viewimage generation process, the image processing unit 10 also acquires thebirds-eye view image of each captured image (the front image 40, therear image 45, the right image 50, and the left image 55).

Next, the image processing unit 10 determines whether or not a capturedimage is present in which a disturbance has occurred (step S315). Here,the disturbance refers to any of the light sources, rain drops, dirt,and the like that have been determined in the disturbance determinationprocess. The image processing unit 10 determines that a disturbance hasoccurred if even one of these disturbances is present.

When determined that a captured image in which a disturbance hasoccurred is not present (NO at step S315), the image processing unit 10uniformly sets the usability (usage rate) in the overlapping visualfield areas of all images to a uniform 50% (step S320). In other words,as shown in FIG. 7, boundary lines BL1 to BL 4 are drawn in thebird's-eye view of the own vehicle VE. The boundary lines BL1 to BL 4are at a 45-degree angle to the advancing direction of the own vehicleVE, with the four corners of the own vehicle VE serving as therespective axes.

Then, the image processing unit 10 changes the images to be used, usingthe boundary lines BL1 to BL 4 as the boundaries. For example, in theoverlapping visual field area (42 and 51) to the front-right of the ownvehicle VE, the front image 40 is used for the area further to the frontside of the own vehicle VE from the boundary line BL2. The right image50 is used for the area further to the right side of the own vehicle VEfrom the boundary line BL2.

In addition, the boundary line BL1 to BL 4 can be changed to anarbitrary angle. For example, as shown in FIG. 7, in the overlappingvisual field area (42 and 51), the usability of the front image 40becomes 0% when the boundary line BL2 is set to be parallel with theadvancing direction of the own vehicle VE. The usability of the rightimage 50 becomes 100%. As still another example, in the overlappingvisual field area (42 and 51), the usability of the front image 40becomes 100% when the boundary line is set to be perpendicular to theadvancing direction of the own vehicle VE. The usability of the rightimage 50 becomes 0%.

When the process at step S320, such as that described above, completed,the image processing unit 10 proceeds to the process at step S350,described hereafter. When determined that an image in which adisturbance has occurred is present in the process at step S315 (YES atstep S315), the image processing unit 10 compares the number of imagesin which a disturbance has occurred with a threshold of the number ofimages that is set in advance (step S325). The threshold of the numberof images can be arbitrarily set based on an application or the likeused to perform the white-line recognition.

When determined that the number of images in which a disturbance hasoccurred is the threshold of the number of images or more (YES at stepS325), the image processing unit 10 sets a diagnosis (flag), indicatingthat white-line recognition is not possible, to ON (step S330). Theimage processing unit 10 then ends the composite bird's-eye view imagegeneration process.

When determined that the number of images in which a disturbance hasoccurred is less than the threshold of the number of images (NO at stepS325), the image processing unit 10 sets the usability of the images inwhich a disturbance has not occurred in the overlapping visual fieldareas to 100% (step S340).

For example, as shown in FIG. 8A, when a light source DT1 is present inthe front image 40 and disturbances are not present in the remainingimages, as shown in FIG. 8B, the image processing unit 10 sets theusability of the front image 40 in the overlapping visual field area to0%. The image processing unit 10 sets the usability of the right image50 and the left image 55 on the front side to 100%. Regarding theoverlapping visual field areas of the rear image 45 with the right image50 and the left image 55, the image processing unit 10 sets theusability to 50%.

Next, the image processing unit 10 sets the usability in the overlappingvisual field areas of images in which disturbances have occurred, basedon the type of disturbance (step S345). According to the presentembodiment, the usability is set to 0% for images in which a disturbanceis present that has a greater effect (disturbance level) on white-linerecognition (also called white-line detection).

In addition, the image processing unit 10 has set the score based on thetype of disturbance in the above-described process. Therefore, in thepresent process, the image processing unit 10 uses this score to set theusability. For example, as shown in FIG. 9A, when dirt DT2 is present inthe front image 40 (−20 points) and a light source DT1 is present in theright image 50 (−30 points), as shown in FIG. 9B, the image processingunit 10 sets the usability of the right image 50 to 0%, the right image50 being the image with the lower score.

Next, the image processing unit 10 generates the birds-eye view image(composite birds-eye view image) based on the usability set for eachoverlapping visual field area. The image processing unit 10 then storesthe bird's-eye view image in the memory 12 (step S250). When a processsuch as this is completed, the image processing unit 10 ends thecomposite birds-eye view image generation process.

Next, the image processing unit 10 returns to the process shown in FIG.3. The image processing unit 10 performs a white-line extraction process(step S135). This process is omitted when the diagnosis (flag),indicating that white-line recognition is not possible, is set to ON. Inaddition, when this process is performed, the image processing unit 10performs a known Hough transform or the like on the bird's-eye viewimage recorded in the memory 12. The image processing unit 10 thenrecords the white-line recognition result and accuracy (reliabilitylevel) in the memory 12. Here, the accuracy of the recognition result isdetermined based on the number of edges (a portion in which thedifference in brightness value is a predetermined amount or more betweenadjacent pixels), the alignment and regularity of the edges, thedifference between the edges and a reference road width that is set inadvance, and the like.

Next, the image processing unit 10 performs a display process (stepS155). In the display process, the image processing unit 10 generates asignal to perform display based on the white-line recognition result(e.g., coordinate) and the accuracy thereof. The image processing unit10 then transmits the signal to the display 26 and the indicator 27.

When a process such as this is completed, the image processing unit 10ends the white-line recognition process.

[Effects]

In the image display system 1 described in detail above, the imageprocessing unit 10 acquires each of the captured images that have beencaptured by the plurality of cameras 21 to 24. The image processing unit10 determines a disturbance level for each of the plurality of capturedimages that have been acquired. The disturbance level indicates whetheror not a disturbance is present in the captured image, or the extent ofthe disturbance.

In addition, regarding an overlapping area in which the imaging areas ofa plurality of captured images overlap, the image processing unit 10selects one or more captured images from the plurality of capturedimages. The image processing unit 10 makes the selection so that thearea occupied by a captured image having a higher disturbance levelamong the plurality captured images is smaller. The image processingunit 10 then generates a composite image based on the plurality ofcaptured images. The image processing unit 10 generates the compositeimage using the captured image selected based on the disturbance levelin the overlapping area.

In the image display system 1 such as this, the proportion occupied byan image in which a disturbance has occurred can be reduced based on thepresence of the disturbance and the extent of the disturbance.Therefore, a composite image can be generated that facilitates checkingof the periphery of the own vehicle VE.

In addition, in the above-described image display system 1, the imageprocessing unit 10 determines the type of disturbance as at least a partof the disturbance level. The image processing unit 10 makes thedetermination based on the brightness value of each pixel composing thecaptured image. The image processing unit 10 selects the captured imageto be used in an overlapping area based on the disturbance level (typeof disturbance).

In the image display system 1 such as this, the type of disturbance isdetermined. Therefore, the captured image to be displayed in theoverlapping area can be more appropriately selected based on the type ofdisturbance.

Furthermore, in the above-described image display system 1, the imageprocessing unit 10 selects a captured image that has the lowestdisturbance level in the overlapping area.

In the image display system 1 such as this, the captured image that hasthe lowest disturbance level is selected. Therefore, the image havingthe highest reliability can be displayed in the overlapping area.

Moreover, in the above-described image display system 1, the imageprocessing unit 10 generates, as the composite image, a bird's-eye viewimage that is supposed to be viewed from directly above the own vehicleVE.

In the image display system 1 such as this, a birds-eye view image canbe provided. Therefore, a process that takes distortion intoconsideration can be omitted when the road surface paint is extracted.Therefore, the process for extracting road surface paint from acomposite image can be simplified.

In addition, in the above-described image display system 1, the imageprocessing unit 10 extracts the road surface paint from the compositeimage.

In the image display system 1 such as this, the road surface paint isextracted from the composite image that has been obtained by theabove-described configuration. Therefore, the road surface paint can beaccurately extracted.

Furthermore, in the above-described image display system 1, the imageprocessing unit 10 calculates the proportion of captured images in whicha disturbance has occurred, among the plurality of captured images. Theimage processing unit 10 performs the calculation based on thedisturbance levels. When determined that the proportion is a thresholdor higher, the image processing unit 10 prohibits the operation forextracting road surface paint. The threshold used here is set inadvance.

In the image display system 1 such as this, when the proportion ofcaptured images in which a disturbance has occurred, among the pluralityof captured images, is the pre-set threshold or more, the operation forextracting the road surface paint is prohibited. Therefore, malfunctionscaused by erroneous detection of road surface paint can be suppressed.

Other Embodiments

The interpretation of the present disclosure is not limited in any wayby the above-described embodiment. In addition, the embodiments of thepresent disclosure include an aspect in which a part of theconfiguration according to the above-described embodiment is omitted toan extent that allows the problems to be solved. Furthermore, theembodiments of the present disclosure include an aspect in which theabove-described plurality of embodiments are combined as appropriate.

Moreover, the embodiments of the present disclosure include all aspectsconceivable without departing from the essence of the present disclosurespecified only by the expressions in the scope of claims. In addition,the reference numbers used in the description of the above-describedembodiment are also used as appropriate in the scope of claims. However,the reference numbers are used to facilitate understanding of thepresent disclosure according to each claim, and are not intended tolimit the technical scope of the present disclosure according to eachclaim.

For example, in the process at S345 according to the above-describedembodiment, the usability of the image that has the higher disturbancelevel in the overlapping visual field area is set to 0%. However, anoccupancy ratio of each captured image in the overlapping area may beset based on a ratio of the numeric values of the disturbance levels.For example, as shown in FIG. 9A, dirt is present in the front image 40(−20 points). A light source is present in the right image 50 (−30points). Therefore, the score of the front image 40 is 80 points. Thescore of the right image 50 is 70 points. Thus, the usability of thefront image 40 can be set to 53.3% and the usability of the right image50 can be set to 46.7%.

In the image display system such as this, the occupancy ratio in of eachcaptured image the overlapping area can be set based on the ratio of thenumeric values of the disturbance levels. Therefore, a highly reliableimage that covers a wider area can be displayed.

[Correspondence Relationship]

The image processing unit 10 according to the above-described embodimentcorresponds to composite image generation apparatus of the presentdisclosure. In addition, among the processes performed by the imageprocessing unit 10 according to the above-described embodiment, theprocess at step S110 corresponds to captured image acquiring means ofthe present disclosure. The process performed at step S120 according tothe above-described embodiment corresponds to disturbance leveldetermining means of the present disclosure.

Furthermore, the processes at S340 and S345 according to theabove-described embodiment correspond to captured image selecting meansof the present disclosure. The process at S350 according to theabove-described embodiment corresponds to image generating means of thepresent disclosure. In addition, the process at S135 according to theabove-described embodiment corresponds to road surface paint extractingmeans of the present disclosure. The process at S325 according to theabove-described embodiment corresponds to disturbance proportioncalculating means of the present disclosure.

Furthermore, the process at S330 according to the above-describedembodiment corresponds to prohibiting means of the present disclosure.

What is claimed is:
 1. A composite image generation apparatus that ismounted in an own vehicle and generates a composite image which is acombination of captured images acquired by a plurality of imaging units,the composite image generation apparatus comprising: a processorconfigured to acquire each of captured images that have been captured bythe plurality of imaging units; determine a disturbance level for eachof the plurality of captured images that have been acquired, thedisturbance level indicating an extent of a disturbance; select, in anoverlapping area in which imaging areas of the plurality of capturedimages overlap, one or more captured images from the plurality ofcaptured images so that the area occupied by a captured image having ahigher disturbance level among the plurality captured images is smaller;and generate a composite image based on the plurality of capturedimages, and generate the composite image by using the captured imageselected based on the disturbance level in the overlapping area, whereinthe processor is configured to deduct a number of points from anallotted score corresponding to each of the plurality of captured imagesin response to the disturbance level.
 2. The composite image generationapparatus according to claim 1, wherein the processor is configured todetermine the type of disturbance as at least a part of the disturbancelevel based on a brightness value of each pixel composing each of thecaptured images; and select one or more captured images from theplurality of captured images based on the disturbance level.
 3. Thecomposite image generation apparatus according to claim 2, wherein theprocessor is configured to select one or more captured images that hasthe lowest disturbance level.
 4. The composite image generationapparatus according to claim 3, wherein the processor is configured to:determine the disturbance level expressed by numeric values; and set anoccupancy ratio of each of the captured images in the overlapping areabased on a ratio of the disturbance level expressed by numeric values ofeach of the captured images.
 5. The composite image generation apparatusaccording to claim 4, wherein the processor is configured to generate,as the composite image, a bird's-eye view image that is supposed to beviewed from directly above the own vehicle.
 6. The composite imagegeneration apparatus according to claim 5, wherein the processor isfurther configured to extract a road surface paint from the compositeimage.
 7. The composite image generation apparatus according to claim 6,wherein the processor is further configured to calculate, based on thedisturbance level, a proportion of captured images in which adisturbance has occurred, among the plurality of captured images; andprohibit an operation of the processor that extracts road surface paintwhen determined that the proportion is a threshold or higher.
 8. Thecomposite image generation apparatus according to claim 1, wherein theprocessor is configured to select one or more captured images that hasthe lowest disturbance level.
 9. The composite image generationapparatus according to claim 1, wherein the processor is configured todetermine the disturbance level expressed by numeric values; and set anoccupancy ratio of each of the captured images in the overlapping areabased on a ratio of the disturbance level expressed by numeric values ofeach of the captured images.
 10. The composite image generationapparatus according to claim 1, wherein the processor is configured togenerate, as the composite image, a bird's-eye view image that issupposed to be viewed from directly above the own vehicle.
 11. Acomposite image generation apparatus that is mounted in an own vehicleand generates a composite image which is a combination of capturedimages acquired by a plurality of imaging units, the composite imagegeneration apparatus comprising: a processor configured to acquire eachof captured images that have been captured by the plurality of imagingunits; determine a disturbance level for each of the plurality ofcaptured images that have been acquired, the disturbance levelindicating an extent of a disturbance; select, in an overlapping area inwhich imaging areas of the plurality of captured images overlap, one ormore captured images from the plurality of captured images so that thearea occupied by a captured image having a higher disturbance levelamong the plurality captured images is smaller; and generate a compositeimage based on the plurality of captured images, and generate thecomposite image by using the captured image selected based on thedisturbance level in the overlapping area, wherein the processor isfurther configured to extract a road surface paint from the compositeimage, and the processor is further configured to calculate, based onthe disturbance level, a proportion of captured images in which adisturbance has occurred, among the plurality of captured images; andprohibit an operation of the processor that extracts road surface paintwhen determined that the proportion is a threshold or higher.
 12. Animage display system comprising: a plurality of imaging units that ismounted in an own vehicle and acquires captured images; a compositeimage generation apparatus that is mounted in the own vehicle andgenerates a composite image which is a combination of captured imagesacquired by the plurality of imaging units; and a display that ismounted in the own vehicle and displays the composite image generated bythe composite image generation apparatus, the composite image generationapparatus comprising a processor configured to acquire each of capturedimages that have been captured by the plurality of imaging units;determine a disturbance level for each of the plurality of capturedimages that have been acquired, the disturbance level indicating anextent of a disturbance; select, in an overlapping area in which imagingareas of the plurality of captured images overlap, one or more capturedimages from the plurality of captured images so that the area occupiedby a captured image having a higher disturbance level among theplurality captured images is smaller; and generate a composite imagebased on the plurality of captured images, and generate the compositeimage by using the captured image selected based on the disturbancelevel in the overlapping area, wherein the processor is configured todeduct a number of points from an allotted score corresponding to eachof the plurality of captured images in response to the disturbancelevel.
 13. The image display system according to claim 12, wherein theplurality of imaging units are a plurality of cameras mounted to the ownvehicle.
 14. The image display system according to claim 12, wherein theprocessor of the composite image generating apparatus determines thedisturbance level for each of the plurality of captured images bydetermining a brightness value of each pixel that composes each of theplurality of captured images.
 15. The image display system according toclaim 14, wherein the processor of the composite image generatingapparatus determines a type of disturbance based on the brightness valueof each pixel that composes each of the plurality of captured images.16. The image display system according to claim 14, wherein theprocessor determines whether the brightness value is saturated.
 17. Theimage display system according to claim 2, wherein the processordetermines whether the brightness value is saturated.
 18. A compositeimage generation apparatus that is mounted in an own vehicle andgenerates a composite image which is a combination of captured imagesacquired by a plurality of imaging units, the composite image generationapparatus comprising: a processor configured to acquire each of capturedimages that have been captured by the plurality of imaging units;determine a disturbance level for each of the plurality of capturedimages that have been acquired, the disturbance level indicating anextent of a disturbance; select, in an overlapping area in which imagingareas of the plurality of captured images overlap, one or more capturedimages from the plurality of captured images so that the area occupiedby a captured image having a higher disturbance level among theplurality captured images is smaller; and generate a composite imagebased on the plurality of captured images, and generate the compositeimage by using the captured image selected based on the disturbancelevel in the overlapping area, wherein the processor is furtherconfigured to determine the type of disturbance as at least a part ofthe disturbance level based on a brightness value of each pixelcomposing each of the captured images; and select one or more capturedimages from the plurality of captured images based on the disturbancelevel, and the processor is further configured to select one or morecaptured images that has the lowest disturbance level, the processor isfurther configured to determine the disturbance level expressed bynumeric values; and set an occupancy ratio of each of the capturedimages in the overlapping area based on a ratio of the disturbance levelexpressed by numeric values of each of the captured images, theprocessor is further configured to generate, as the composite image, abird's-eye view image that is supposed to be viewed from directly abovethe own vehicle, the processor is further configured to extract a roadsurface paint from the composite image, and the processor is furtherconfigured to calculate, based on the disturbance level, a proportion ofcaptured images in which a disturbance has occurred, among the pluralityof captured images; and prohibit an operation of the processor thatextracts road surface paint when determined that the proportion is athreshold or higher.